The present invention relates to a method for coding and/or decoding a video signal for digital storage media and transmission media.
A Lattice Vector Quantizer (LVQ) has been widely used in many systems of digital video signal bandwidth compression. A lattice is defined as a set of vectors A={x: x=U1.times.A1+U2.times.A2+. . . +UN.times.AN} where An (n=1, 2, . . . , N) are the basis vectors of the lattice and Un are integers. In a particular application, LVQ assumes truncation and scaling of the lattice and devising a method to quantize vectors that fall outside the truncated lattice. Truncation means selecting L lattice points x to be the output points Yi so that each source vector x is replaced by a nearest lattice point Yi.
The truncation and scaling is usually accomplished for a fixed number of lattice vectors L by trying various Euclidean distance scale factors on vectors generated from appropriate distributions and choosing the scale factor which minimizes the mean squared error. This distribution should be known prior to the encoding process. To know where to truncate the lattice for a given L lattice vectors, the number of lattice vectors in shells must be calculated at various radial distances from the origin of distribution. Such algorithms of calculation is known only for symmetrical distribution, like Gaussian or Laplacian distribution.
In practice, a real distribution of source vectors can be completely different. Incorrect truncation will always produce two well known distortions--overload quantization noise and granular quantization noise. The former is due to clipping the vectors outside the truncation border, the latter relates to rounding of vector components inside the truncation area. The problem is to find a balance between the two.
Another problem is that no algorithm exists for calculating L for any type of distribution. This means that by using the mentioned distributions instead of a real value, the LVQ will produce redundant number of lattice vectors L that increase the volume of code book and place a burden on system resources.
Usually, the conventional methods use fixed truncation and scaling procedures. On the other hand, a fixed LVQ should be able to simultaneously ensure a large range to quantize vectors with large norm or contrast and little distortion for vectors with low contrast. This contradictory demand is the main obstacle to improving the performance of conventional methods. Usually, attempts to increase compression by lowering L lead to fast decreasing of visual quality of a decoded video signal.